Deep Sketch Hashing: Fast Free-hand Sketch-Based Image Retrieval

Free-hand sketch-based image retrieval (SBIR) is a specific cross-view retrieval task, in which queries are abstract and ambiguous sketches while the retrieval database is formed with natural images. Work in this area mainly focuses on extracting representative and shared features for sketches and natural images. However, these can neither cope well with the geometric distortion between sketches and images nor be feasible for large-scale SBIR due to the heavy continuous-valued distance computation. In this paper, we speed up SBIR by introducing a novel binary coding method, named \textbf{Deep Sketch Hashing} (DSH), where a semi-heterogeneous deep architecture is proposed and incorporated into an end-to-end binary coding framework. Specifically, three convolutional neural networks are utilized to encode free-hand sketches, natural images and, especially, the auxiliary sketch-tokens which are adopted as bridges to mitigate the sketch-image geometric distortion. The learned DSH codes can effectively capture the cross-view similarities as well as the intrinsic semantic correlations between different categories. To the best of our knowledge, DSH is the first hashing work specifically designed for category-level SBIR with an end-to-end deep architecture. The proposed DSH is comprehensively evaluated on two large-scale datasets of TU-Berlin Extension and Sketchy, and the experiments consistently show DSH's superior SBIR accuracies over several state-of-the-art methods, while achieving significantly reduced retrieval time and memory footprint.Comment: This paper will appear as a spotlight paper in CVPR201

Why does the apparent mass of a coronal mass ejection increase?

Mass is one of the most fundamental parameters characterizing the dynamics of a coronal mass ejection (CME). It has been found that CME apparent mass measured from the brightness enhancement in coronagraph images shows an increasing trend during its evolution in the corona. However, the physics behind it is not clear. Does the apparent mass gain come from the mass outflow from the dimming regions in the low corona, or from the pileup of the solar wind plasma around the CME when it propagates outwards from the Sun? We analyzed the mass evolution of six CME events. Their mass can increase by a factor of 1.6 to 3.2 from 4 to 15 Rs in the field of view (FOV) of the coronagraph on board the Solar Terrestrial Relations Observatory (STEREO). Over the distance about 7 to 15 Rs, where the coronagraph occulting effect can be negligible, the mass can increase by a factor of 1.3 to 1.7. We adopted the `snow-plough' model to calculate the mass contribution of the piled-up solar wind in the height range from about 7 to 15 Rs. For 2/3 of the events, the solar wind pileup is not sufficient to explain the measured mass increase. In the height range from about 7 to 15 Rs, the ratio of the modeled to the measured mass increase is roughly larger than 0.55. Although the ratios are believed to be overestimated, the result gives evidence that the solar wind pileup probably makes a non-negligible contribution to the mass increase. It is not clear yet whether the solar wind pileup is a major contributor to the final mass derived from coronagraph observations. However, our study suggests that the solar wind pileup plays increasingly important role in the mass increase as a CME moves further away from the Sun.Comment: 27 pages, 2 tables, 9 figures, accepted by Ap